Systems and methods for assisting and augmenting surgical procedures

ABSTRACT

Systems and methods for providing assistance to a surgeon during an implant surgery are disclosed. A method includes defining areas of interest in diagnostic data of a patient and defining a screw bone type based on the surgeon&#39;s input. Post defining the areas of interest, salient points are determined for the areas of interest. Successively, an XZ angle, an XY angle, and a position entry point for a screw are determined based on the salient points of the areas of interest. Successively, a maximum screw diameter and a length of the screw are determined based on the salient points. Thereafter, the screw is identified and suggested to the surgeon for usage during the implant surgery.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/537,869, filed on Jul. 27, 2017 titled “SYSTEMS AND METHODS OFPROVIDING ASSISTANCE DURING A SPINAL SURGERY,” which is hereinincorporated by reference in its entirety.

FIELD OF THE DISCLOSURE

The present disclosure is generally related to providing surgicalassistance to a surgeon, and more particularly for providing surgicalassistance for a surgical procedure.

BACKGROUND

Assessing spinal deformity is of tremendous importance for a number ofdisorders affecting human spine. A pedicle is a dense stem-likestructure that projects from the posterior of a vertebra. There are twopedicles per vertebra that connect to structures like a lamina and avertebral arch. Conventionally available screws, used in spinalsurgeries, are poly-axial pedicle screws made of titanium. Titanium ischosen as it is highly resistant to corrosion and fatigue, and is easilyvisible in MRI images.

The pedicle screws were originally placed via a free-hand technique.Surgeons performing spinal surgeries merely rely on their experience andknowledge of known specific paths for performing the spinal surgeries.The free-hand techniques used by spinal surgeons rely on spinal anatomyof a patient. The spinal surgeon relies on pre-operative imaging andintra-operative anatomical landmarks for performing the spinal surgery.Assistive fluoroscopy and navigation are helpful in that they guidepedicle screw placement more or less in a real-time, but are limited bytime and costs involved in fluoroscopy, and significant radiationexposure during fluoroscopy.

The subject matter discussed in the background section should not beassumed to be prior art merely as a result of its mention in thebackground section. Similarly, a problem mentioned in the backgroundsection or associated with the subject matter of the background sectionshould not be assumed to have been previously recognized in the priorart. The subject matter in the background section merely representsdifferent approaches, which in and of themselves may also correspond toimplementations of the claimed technology.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate various embodiments of systems,methods, and embodiments of various other aspects of the disclosure. Anyperson with ordinary skills in the art will appreciate that theillustrated element boundaries (e.g. boxes, groups of boxes, or othershapes) in the figures represent one example of the boundaries. It maybe that in some examples one element may be designed as multipleelements or that multiple elements may be designed as one element. Insome examples, an element shown as an internal component of one elementmay be implemented as an external component in another, and vice versa.Furthermore, elements may not be drawn to scale. Non-limiting andnon-exhaustive descriptions are described with reference to thefollowing drawings. The components in the figures are not necessarily toscale, emphasis instead being placed upon illustrating principles.

FIG. 1A illustrates a system for providing assistance prior to or duringan implant surgery, according to an embodiment.

FIG. 1B illustrates a network connection diagram 100 of an implantsurgery assistance system for providing assistance prior to or during animplant surgery, according to an embodiment.

FIG. 2 illustrates a block diagram showing components of an implantsurgery assistance system, according to an embodiment.

FIG. 3A shows salient points presented in a top view of a vertebra ofthe patient, according to an embodiment

FIG. 3B shows salient points present in a side view of the vertebra ofthe patient, according to an embodiment.

FIG. 4A illustrates computations for determining an XZ angle for aspinal screw, according to an embodiment.

FIG. 4B illustrates computations for determining an XY angle for aspinal screw, according to an embodiment.

FIG. 5 illustrates a spinal screw and dimensions of the spinal screw,according to an embodiment.

FIG. 6 illustrates a flowchart showing a method for providing assistanceto the surgeon during the spinal surgery, according to an embodiment.

FIG. 7 illustrates a flowchart showing a method for generating implantconfigurations.

FIG. 8A illustrates a flowchart showing a method for applying analysisprocedures that can utilize machine learning models, according to anembodiment.

FIG. 8B illustrates a flowchart showing a method for applying analysisprocedures that can utilize virtual models, according to an embodiment.

FIG. 9 illustrates a flowchart showing a method for training a machinelearning model, according to an embodiment.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described more fullyhereinafter with reference to the accompanying drawings in which likenumerals represent like elements throughout the several figures, and inwhich example embodiments are shown. Embodiments of the claims may,however, be embodied in many different forms and should not be construedas limited to the embodiments set forth herein. The examples set forthherein are non-limiting examples and are merely examples among otherpossible examples.

The words “comprising,” “having,” “containing,” and “including,” andother forms thereof, are intended to be equivalent in meaning and beopen ended in that an item or items following any one of these words isnot meant to be an exhaustive listing of such item or items, or meant tobe limited to only the listed item or items.

It must also be noted that as used herein and in the appended claims,the singular forms “a,” “an,” and “the” include plural references unlessthe context clearly dictates otherwise. Although any systems and methodssimilar or equivalent to those described herein can be used in thepractice or testing of embodiments of the present disclosure, thepreferred, systems and methods are now described.

FIG. 1A illustrates a system 152 for providing assistance prior to orduring an implant surgery, according to an embodiment. The system 152can improve surgeries that involve implants by guiding selection andapplication of implants, delivery instruments, navigation tools, or thelike. The system 152 can comprise hardware components that improvesurgeries using, for example, a surgical assistance system 164. Invarious implementations, the surgical assistance system 164 can obtainimplant surgery information, converting the implant surgery informationinto a form compatible with an analysis procedure, applying the analysisprocedure to obtain results, and using the results to provide aconfiguration for the implant surgery.

An implant configuration can include characteristics of an implant suchas various dimensions, angles, materials, application features (e.g.,implant sizes, implant functionality, anchoring features, suture type,etc.), and/or aspects of applying the implant such as insertion point,delivery path, implant position/angle, rotation, amounts of force toapply (e.g., torque applied to a screw, rotational speed of a screw,rate of expansion of expandable implants, and so forth), etc. In someimplementations, the implant surgery information can include images of atarget area, such as MRI scans of a spine, patient information such assex, weight, etc., or a surgeon's pre-operative plan. The surgicalassistance system 164 can convert the implant surgery information, forexample, by converting images into arrays of integers or histograms,entering patient information into feature vectors, or extracting valuesfrom the pre-operative plan.

In some implementations, surgical assistance system 164 can analyze oneor more images of a patient to identify one or more features ofinterest. The features of interest can include, without limitation,implantation sites, targeted features, non-targeted features, accesspaths, anatomical structures, or combinations thereof. The implantationsites can be determined based upon one or more of risk factors, patientinformation, surgical information, or combinations thereof. The riskfactors can be determined by the surgical assistant system based uponthe patient's medical history. For example, if the patient issusceptible to infections, the surgical assistant system 164 canrecommend a minimally invasive procedure whereas the surgical assistantsystem may recommend open procedure access paths for patients lesssusceptible to infection. In some implementations, the physician canprovide the risk factors before or during the procedure. Patientinformation can include, without limitation, patient sex, age, healthrating, or the like. The surgical information can include availablenavigation systems, robotic surgery platforms, access tools, surgerykits, or the like.

In some implementations, surgical assistance system 164 can applyanalysis procedures by supplying the converted implant surgeryinformation to a machine learning model trained to select implantconfigurations. For example, a neural network model can be trained toselect pedicle screw configurations for a spinal surgery. The neuralnetwork can be trained with training items each comprising a set ofimages scans (e.g. camera, MRI, CT, x-ray, etc.) and patientinformation, an implant configuration used in the surgery, and/or ascored surgery outcome resulting from one or more of: surgeon feedback,patient recovery level, recovery time, results after a set number ofyears, etc. This neural network can receive the converted surgeryinformation and provide output indicating the pedicle screwconfiguration.

In other implementations, surgical assistance system 164 can apply theanalysis procedure by A) localizing and classifying a surgery target, B)segmenting the target to determine boundaries, C) localizing optimalimplant insertion points, D) identifying target structures (e.g.pedicles and isthmus), and/or computing implant configurations based onresults of A-D.

In yet further implementations, surgical assistance system 164 can applythe analysis procedure by building a virtual model of a surgery targetarea, localizing and classifying areas of interest within the virtualmodel, segmenting areas of interest, localizing insertion points, andcomputing implant configurations by simulating implant insertions in thevirtual model. Each of the individual steps of these implementations canbe accomplished using a machine learning model trained (as discussedbelow) to identify appropriate results for that step or by applying acorresponding algorithm. For example, an algorithm can measure anisthmus by determining an isthmus width in various images and trackingthe minimal value across the images in different planes.

In another example, surgical assistance system 164 can apply theanalysis procedure by performing a finite element analysis on agenerated three-dimensional model (e.g., a model of the patient'sanatomy) to assess stresses, strains, deformation characteristics (e.g.,load deformation characteristics), fracture characteristics (e.g.,fracture toughness), fatigue life, etc. A virtual representation of theimplant or other devices could be generated. The surgical assistancesystem 164 can generate a three-dimensional mesh to analyze the model.Machine learning techniques to create an optimized mesh based on adataset of vertebrae or other bones and implants or other devices. Afterperforming the analysis, the results could be used to refine theselection of screws or other implant components.

The surgical assistance system 164 can incorporate results from theanalysis procedure in suggestions for the implant surgery. For example,the results can be used to indicate suggested implants for a procedure,to annotate an image with suggested insertions points and angles, togenerate a virtual reality or augmented reality representation includingthe suggested implant configurations, to provide warnings or otherfeedback to surgeons during a procedure, to automatically order thenecessary implants, to generate surgical technique information (e.g.,insertion forces/torques, imaging techniques, delivery instrumentinformation, or the like), etc.

The surgical assistance system 164 can improve the efficiency,precision, and/or efficacy of implant surgeries by providing moreoptimal implant configuration guidance. This can reduce operationalrisks and costs produced by surgical complications, reduce the resourcesrequired for preoperative planning efforts, and reduce the need forextensive implant variety to be prepared prior to an implant surgery.The surgical assistance system 164 provides increased precision andefficiency for patients and surgeons.

In orthopedic surgeries, the surgical assistance system 164 can selector recommend implants (e.g., permanent implants, removable implants,etc.), surgical techniques, patient treatment plans, or the like. Forexample, the implants can be joint replacements, hip implants, removablebone screws, or the like. The surgical techniques can include accessinstruments selected based on one or more criteria, such as risk ofadverse events, optical implant position, protected zones (e.g., zoneswith nerve tissue), or the like. In spinal surgeries, the surgicalassistance system 164 can reduce incorrect selection of pedicle screwtypes, dimensions, and trajectories while making surgeons more efficientand precise, as compared to existing surgical procedures.

The surgical assistance system 164 can also improve surgicalrobotics/navigation systems, providing improved intelligence forselecting implant application parameters. For example, the surgicalassistance system 164 empowers surgical robots and navigation systemsfor spinal surgeries to increase procedure efficiency and reduce surgeryduration by providing information on types and sizes, along withexpected insertion angles. In addition, hospitals benefit from reducedsurgery durations and reduced costs of purchasing, shipping, and storingalternative implant options. Medical imaging and viewing technologiescan integrate with the surgical assistance system 164, to provide moreintelligent and intuitive results.

The surgical assistance system 164 can be incorporated in system 152,which can include one or more input devices 120 that provide input tothe processor(s) 145 (e.g. CPU(s), GPU(s), HPU(s), etc.), notifying itof actions. The actions can be mediated by a hardware controller thatinterprets the signals received from the input device and communicatesthe information to the processors 145 using a communication protocol.Input devices 120 include, for example, a mouse, a keyboard, atouchscreen, an infrared sensor, a touchpad, a wearable input device, acamera- or image-based input device, a microphone, or other user inputdevices.

Processors 145 can be a single processing unit or multiple processingunits in a device or distributed across multiple devices. Processors 145can be coupled to other hardware devices, for example, with the use of abus, such as a PCI bus or SCSI bus. The processors 145 can communicatewith a hardware controller for devices, such as for a display 130.Display 130 can be used to display text and graphics. In someimplementations, display 130 provides graphical and textual visualfeedback to a user. In some implementations, display 130 includes theinput device as part of the display, such as when the input device is atouchscreen or is equipped with an eye direction monitoring system. Insome implementations, the display is separate from the input device.Examples of display devices are: an LCD display screen, an LED displayscreen, a projected, holographic, or augmented reality display (such asa heads-up display device or a head-mounted device), and so on. OtherI/O devices 140 can also be coupled to the processor, such as a networkcard, video card, audio card, USB, firewire or other external device,camera, printer, speakers, CD-ROM drive, DVD drive, disk drive, orBlu-Ray device. Other I/O 140 can also include input ports forinformation from directly connected medical equipment such as MRImachines, X-Ray machines, etc. Other I/O 140 can further include inputports for receiving data from these types of machine from other sources,such as across a network or from previously captured data, e.g. storedin a database.

In some implementations, the system 152 also includes a communicationdevice capable of communicating wirelessly or wire-based with a networknode. The communication device can communicate with another device or aserver through a network using, for example, TCP/IP protocols. System152 can utilize the communication device to distribute operations acrossmultiple network devices.

The processors 145 can have access to a memory 150 in a device ordistributed across multiple devices. A memory includes one or more ofvarious hardware devices for volatile and non-volatile storage, and caninclude both read-only and writable memory. For example, a memory cancomprise random access memory (RAM), various caches, CPU registers,read-only memory (ROM), and writable non-volatile memory, such as flashmemory, hard drives, floppy disks, CDs, DVDs, magnetic storage devices,tape drives, device buffers, and so forth. A memory is not a propagatingsignal divorced from underlying hardware; a memory is thusnon-transitory. Memory 150 can include program memory 160 that storesprograms and software, such as an operating system 162, surgicalassistance system 164, and other application programs 166. Memory 150can also include data memory 170 that can include, e.g. implant surgeryinformation, configuration data, settings, user options or preferences,etc., which can be provided to the program memory 160 or any element ofthe system 152.

Some implementations can be operational with numerous other computingsystem environments or configurations. Examples of computing systems,environments, and/or configurations that may be suitable for use withthe technology include, but are not limited to, personal computers,server computers, handheld or laptop devices, cellular telephones,wearable electronics, tablet devices, multiprocessor systems,microprocessor-based systems, programmable consumer electronics, networkPCs, minicomputers, mainframe computers, distributed computingenvironments that include any of the above systems or devices, or thelike.

FIG. 1B illustrates a network connection diagram 100 of a system 102 forproviding assistance to a surgeon during a spinal surgery, according toan embodiment. The system 102 may be connected to a communicationnetwork 104. The communication network 104 may further be connected witha network in the form of a precision spine network 106 for allowing datatransfer between the system 102 and the precision spine network 106.

The communication network 104 may be a wired and/or a wireless network.The communication network 104, if wireless, may be implemented usingcommunication techniques such as Visible Light Communication (VLC),Worldwide Interoperability for Microwave Access (WiMAX), Long termevolution (LTE), Wireless local area network (WLAN), Infrared (IR)communication, Public Switched Telephone Network (PSTN), Radio waves,and other communication techniques known in the art.

In one embodiment, the precision spine network 106 may be implemented asa facility over “the cloud” and may include a group of modules. Thegroup of modules may include a Precision Spine Network Base (PSNB)module 108, an abnormalities module 110, an XZ screw angle module 112,an XY screw module 114, and a screw size module 116.

The PSNB module 108 may be configured to store images of patients andtypes of spinal screws, required in spinal surgeries. In someimplementations, a similar module can be used for other types ofsurgeries. While the PSNB is referred to below, in each instance othersimilar modules can be used for other types of surgeries. For example, aPrecision Knee Network Based can be used to assist in anterior cruciateligament (ACL) replacement surgeries. The images may be any of cameraimages, Magnetic Resonance Imaging (MRI) images, ultrasound images,Computerized Aided Tomography (CAT) scan images, Positron EmissionTomography (PET) images, and X-Ray images. In one case, the images maybe analyzed to identify abnormalities and salient features in theimages, for performing spinal surgeries on the patients. In someimplementations, the PSNB module 108 can store additional implantsurgery information, such as patient information, (e.g. sex, age,height, weight, type of pathology, occupation, activity level, tissueinformation, health rating, etc.), specifics of implant systems (e.g.types and dimensions), availability of available implants, aspects of asurgeon's preoperative plan (e.g. surgeon's initial implantconfiguration, detection and measurement of the patient's anatomy onimages, etc.), etc. In some implementations, the PSNB module 108 canconvert the implant surgery information into formats useable for implantsuggestion models and algorithms. For example, the implant surgeryinformation can be tagged with particular identifiers for formulas orcan be converted into numerical representations suitable for supplyingto a machine learning model.

The abnormalities module 110 may measure distances between a number ofsalient features of one vertebra with salient features of anothervertebra, for identifying disk pinches or bulges. Based on theidentified disk pinches or bulges, herniated disks may be identified inthe patients. It should be obvious to those skilled in the art, thatgiven a wide variety of salient features and geometric rules, manyspinal abnormalities could be identified. If the spinal abnormalitiesare identified, the PSNB module 108 may graphically identify areashaving the spinal abnormalities and may send such information to a userdevice 118.

In one embodiment, information related to spinal surgeries may bedisplayed through a Graphical User Interface (GUI) of the user device118, as illustrated using FIG. 1B. A smart phone is shown as the userdevice 118 in FIG. 1B, as an example. Further, the user device 118 maybe any other device including a GUI, for example, a laptop, desktop,tablet, phablet, or other such devices known in the art.

The XZ screw angle module 112 may determine an XZ angle of a spinalscrew or other implant to be used during the surgery. Further, the XYscrew angle module 114 may determine an XY angle of the implant. The XZscrew angle module 112 and the XY screw angle module 114 may determine aposition entry point for at least one spinal screw. The XZ screw anglemodule 112 and the XY screw angle module 114 may graphically representthe identified data and may send such information to the user device118.

The screw size module 116 may be used to determine a screw diameter(e.g., a maximum screw diameter) and a length of the screw based on thesalient features identified from the images of the patients.

In some implementations, the XZ screw angle module 112, the XY screwangle module 114, and the screw size module 116 can identify implantconfigurations for other types of implants in addition to, or other thanscrews (e.g., pedicle screws, facet screws, etc.) such as cages, plates,rods, disks, fusions devices, spacers, rods, expandable devices, etc. Inaddition, these modules may suggest implant configurations in relationto references other than an X, Y, Z, coordinate system. For example, ina spinal surgery, the suggestions can be in reference to the sagittalplane, mid-sagittal plane, coronal plane, frontal plane, or transverseplane. As another example, in an ACL replacement surgery, thesuggestions can be an angle for a tibial tunnel in reference to thefrontal plane of the femur. In various implementations, the XZ screwangle module 112, the XY screw angle module 114, or screw size module116 can identify implant configurations using machine learning modules,algorithms, or combinations thereof, as described below in relation toFIGS. 6-9.

In one embodiment, referring to FIG. 2, a block diagram showingdifferent components of the system 102 is explained. The system 102includes a processor 202, interface(s) 204, and a memory 206. Theprocessor 202 may execute an algorithm stored in the memory 206 foraugmenting an implant surgery, e.g. by providing assistance to a surgeonduring a spinal or other implant surgery, by providing controls to arobotic apparatus (e.g., robotic surgery systems, navigation system,etc.) for an implant surgery or by generating suggestions for implantconfigurations to be used in an implant surgery. The processor 202 mayalso be configured to decode and execute any instructions received fromone or more other electronic devices or server(s). The processor 202 mayinclude one or more general purpose processors (e.g., INTEL® or AdvancedMicro Devices® (AMD) microprocessors) and/or one or more special purposeprocessors (e.g., digital signal processors or Xilinx® System On Chip(SOC) Field Programmable Gate Array (FPGA) processor). The processor 202may be configured to execute one or more computer-readable programinstructions, such as program instructions to carry out any of thefunctions described in this description.

The interface(s) 204 may help a user to interact with the system 102.The user may be any of an operator, a technician, a doctor, a doctor'sassistant, or another automated system controlled by the system 102. Theinterface(s) 204 of the system 102 may either accept an input from theuser or provide an output to the user, or may perform both the actions.The interface(s) 204 may either be a Command Line Interface (CLI),Graphical User Interface (GUI), or a voice interface.

The memory 206 may include, but is not limited to, fixed (hard) drives,magnetic tape, floppy diskettes, optical disks, Compact Disc Read-OnlyMemories (CD-ROMs), and magneto-optical disks, semiconductor memories,such as ROMs, Random Access Memories (RAMs), Programmable Read-OnlyMemories (PROMs), Erasable PROMs (EPROMs), Electrically Erasable PROMs(EEPROMs), flash memory, magnetic or optical cards, or other type ofmedia/machine-readable medium suitable for storing electronicinstructions.

The memory 206 may include modules, implemented as programmedinstructions executed by the processor 202. In one case, the memory 206may include a design module 208 for receiving information from theabnormalities module 110. The design module 208 may poll the surgeon foran information request. The design module 208 may allow the surgeon todesign the spinal screw and change the generated implant configurations,such as the entry point (e.g., entry point into the patient, entrypoints into a vertebra, entry points to the implantation site, etc.),and screw or other implant angles in any of various planes. If thesurgeon changes the entry point or angles, the system can automaticallyupdate other features of the implant configuration to account for thechanges, such as the implant dimensions (e.g. screw diameter, threadpitch, or length). The design module 208 may include patient data 210.The patient data 210 may include images of patients and may allow thesurgeon to identify the patients. A patient may refer to a person onwhom and operations is to be performed. The patient data 210 may includeimages of patients, received from the user device 118.

In one embodiment, areas of interest may be defined in diagnostic dataof a patient. In one case, the system 102 may determine the areas ofinterest based on pre-defined rules or using machine learning models, asdescribed below in relation to FIGS. 6-9. In another case, the areas ofinterest may be defined based on a surgeon's input. In one case, thediagnostic data may include images of the patient. The images may be anyof camera images, Magnetic Resonance Imaging (MRI) images, ultrasoundimages, Computerized Aided Tomography (CAT) scan images, PositronEmission Tomography (PET) images, and X-Ray images. In one case, theimages of the patients may be stored in the patient surgeon database210.

Post defining the areas of interest, a screw bone type may be definedbased on various models and/or the surgeon's input. Successively,salient features of the areas of interest may be identified in theimages of the patients, e.g. by applying the procedures described below.FIG. 3a shows salient points present in a top view of a vertebra of thepatient, according to an embodiment. The salient points are shown asbubbles i.e. ‘e₁,’ ‘e₂,’ and ‘f₂.’ Further, FIG. 3b shows salient pointspresent in a side view of the vertebra of the patient, according to anembodiment. The salient points are shown as bubbles i.e. ‘k_(i),’‘k_(u),’ ‘h_(u),’ ‘i_(m),’ and ‘z’.'

Successively, based on the salient points of the areas of interest, thesystem 102 may determine implant configurations (e.g. angles and entrypoint, implant orientation, implant movement, etc.) using the analysisprocedures. FIG. 4a illustrates computations for determining the XZangle (Φ) 402 using the salient points, according to an embodiment. Itshould be noted that positions of X and Y co-ordinates of the regions ofinterest may be determined based on a location of at least one salientfeature present in the image.

FIG. 4b illustrates computations for determining the XY angle (θ) 406using the salient points, according to an embodiment. It should be notedthat positions of X and Y co-ordinates of the regions of interest may bedetermined based on a location of at least one salient feature presentin the image. Further, FIG. 4a illustrates a position entry point 404for the spinal screw, and FIG. 4b illustrates a position entry point 408for the spinal screw. Upon determining, MRI data including the XY angle,the XZ angle, and the position entry point for the spinal screw, may bestored in the abnormalities module 110.

Post identification of the angels and the entry point for an implant,the system 102 may determine additional implant configuration features.For example, the system 102 can determine a maximum implant (e.g. spinalscrew) diameter, a minimum implant diameter, and a length of the implantto be used during a spinal surgery. For example, upon determining themaximum spinal screw diameter and the length of the spinal screw, theprocedure MRI data may be updated in the abnormalities module 110.

In the spinal surgery example, the spinal screw having the determinedmaximum screw diameter and the length may be identified. The spinalscrew may be suggested, to the surgeon, for usage during the spinalsurgery. In one case, a spinal screw HA and dimensions of the spinalscrew HA may be illustrated for the surgeon's selection, as shown inFIG. 5. As illustrated in FIG. 5, a schematic showing differentparameters of the spinal screw HA, dimensions of the spinal screw HA,and a schematic of threads of the spinal screw HA are shown, accordingto an embodiment. Further, different such details related to spinalscrews HB, spinal screws HD, and other known spinal screws may bepresented to the surgeon for usage during the spinal surgery, therebyassisting the surgeon during the spinal surgery.

As another example, for an ACL replacement, upon determining the entrypoint and angle for a tibial tunnel for attaching a replacement graft,the system 102 can identify a depth for the tibial tunnel such that itwill end above the center of the knee joint without piercing surroundingtissue. In addition, dimensions for the ACL graft itself and/or forscrews or other fastening components can be suggested.

The flowchart 600 of FIG. 6 shows the architecture, functionality, andoperation for providing assistance to a surgeon during a spinal surgery,according to an embodiment. One skilled in the art will appreciate that,for this and other processes and methods disclosed herein, the functionsperformed in the processes and methods may be implemented in differingorder. Furthermore, the outlined steps and operations are only providedas examples, and some of the steps and operations may be optional,combined into fewer steps and operations, or expanded into additionalsteps and operations without detracting from the essence of thedisclosed embodiments. For example, two blocks shown in succession inFIG. 6 may in fact be executed substantially concurrently or the blocksmay sometimes be executed in the reverse order, depending upon thefunctionality involved. Any process descriptions or blocks in flowchartsshould be understood as representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process, and alternateimplementations are included within the scope of the example embodimentsin which functions may be executed out of order from that shown ordiscussed, including substantially concurrently or in reverse order,depending on the functionality involved. In addition, the processdescriptions or blocks in flow charts should be understood asrepresenting decisions made by a hardware structure such as a statemachine. The flowchart 600 starts at step 602 and concludes at step 610.

At step 602, areas of interest may be defined in diagnostic data of apatient and a screw bone type may be defined, during a spinal surgery.The diagnostic data may include images of the patient. The images may beany of camera images, Magnetic Resonance Imaging (MRI) images,ultrasound images, Computerized Aided Tomography (CAT) scan images,Positron Emission Tomography (PET) images, and X-Ray images.

At step 604, salient features of areas of interest may be identifiedfrom the diagnostic data. In one case, the images may be analyzed toidentify abnormalities and the salient features, for performing spinalsurgeries on the patients.

At step 606, an XZ angle, an XY angle, and a position entry point for animplant (e.g. a spinal screw) are determined. In one case, the XZ angle,the XY angle, and the position entry point may be determined based onthe salient features.

At step 608, a maximum screw diameter and a length of the screw to beused during the spinal surgery may be determined based on the XY angle,the XZ angle, and the position entry point of the screw. Upondetermining the maximum screw diameter and the length of the screw, theprocedure MRI data may be updated in an abnormalities module 110.

At step 610, the screw implant to be used during a surgery may beidentified and suggested to a surgeon. The screw implant may beidentified based on the maximum screw diameter and the length of thescrew.

FIG. 7 illustrates a flowchart showing a method 700 for generatingimplant configurations. At block 702, method 700 can obtain implantsurgery information such as images, patent history, circumstance, testresults, biographic data, surgeon recommendations, implant specifics,etc. Implant surgery images can be of parts of a patient, such as cameraimages, Magnetic Resonance Imaging (MRI) images, ultrasound images,Computerized Aided Tomography (CAT) scan images, Positron EmissionTomography (PET) images, X-Ray images, 2D or 3D virtual models, CADmodels, etc. Additional implant surgery information can include, e.g.sex, age, height, weight, type of pathology, occupation, activity level,implant types and dimensions, availability of available implants, oraspects of a surgeon's preoperative plan (e.g. surgeon's initial implantconfiguration, detection and measurement of the patient's anatomy onimages, etc.)

The implant surgery information can be obtained in various manners suchas through direct user input (e.g. through a terminal or by interactingwith a web service), through automatic interfacing with networkeddatabases (e.g. connecting to patient records stored by a hospital,laboratory, medical data repositories, etc.), by scanning documents, orthrough connected scanning, imaging, or other equipment. The patientdata can be gathered with appropriate consent and safeguards to remainHIPPA compliant.

At block 704, method 700 can convert the implant surgery informationobtained at block 702 to be compatible with analysis procedures. Theconversion can depend on the analysis procedure that will be used. Asdiscussed below in relation to block 706, analysis procedures caninclude directly applying a machine learning model, applying analgorithm with multiple stages where any of the stages can providemachine learning model predictions (see FIG. 8A), or applying a virtualmodeling system (see FIG. 8B).

In some implementations, the conversion of the implant surgeryinformation can include formatting the implant surgery information forentry to a machine learning model. For example, information such aspatient sex, height, weight, etc. can be entered in a feature vector,such as a sparse vector with values corresponding to available patientcharacteristics. In some implementations, the conversions can includetransforming images from the implant surgery information into a formatsuitable for input to a machine learning model, e.g. an array ofintegers representing pixels of the image, histograms, etc. In someimplementations, the conversion can include identifying surgery targetfeatures (detection and measurement of the patient's anatomy on images),characterizing surgery targets, or modeling (i.e. creating a virtualmodel of) the implant surgery target. For example, in a spinal surgery,this can include measuring vertebrae features on an image, converting 2Dimages of vertebrae into a 3D model, or identifying which vertebrae froma series of images are to be the target of the implant operation. Asanother example, in an ACL replacement surgery, this can includeidentifying and measuring features in an image such as location, size,and spacing of anatomy such as of the femur, patella, remaining portionof meniscus, other ligaments, etc., converting 2D images of the kneeinto a 3D model, or identifying other areas of damage (e.g. fractures,torn cartilage, other ligament tears, etc.).

In various implementations, the conversion process can be automatic,human supervised, or performed by a human technician, e.g. using toolssuch as a digital ruler and a digital angle finder. Further in thespinal surgery example, the conversion can include identifying a targetset of vertebrae, initially localizing and marking the target set ofvertebrae, performing segmentation for each of the target set ofvertebrae, and marking cortical boundaries. In some implementations,input for the spinal implant surgery can specify a target set ofvertebrae, however the surgical assistance system 164 can automaticallyperform calculations for additional vertebrae that weren't specified inthe inputs. This can give the surgeon an option to expand the set ofvertebrae to be fused, either prior to the operation or even during theprocedure. In the ACL replacement surgery example, the conversion caninclude identifying a graft type (e.g. patella tendon, hamstring,cadaver ACL, etc.), initially localizing or marking the target drillingsites, performing segmentation for the target features (e.g. end of thefemur), and marking boundaries (e.g. bone edges, meniscus edges,synovial membrane, etc.).

At block 706, method 700 can apply analysis procedures, using theconverted implant surgery information from block 704, to identifyimplant configuration(s). In various implementations, the analysisprocedures can include directly applying a machine learning model,applying a sequence of steps that can include one or more machinelearning models, and/or generating a virtual model of the surgery targetarea and applying heuristics for implant configuration selection.

To apply a machine learning model directly, method 700 can provide theconverted implant information to a machine learning model trained tospecify implant configurations. A machine learning model can be trainedto take input such as representations of a series of images and afeature vector for the patient and other aspects of the surgery (e.g.implant availability, surgeon specialties or ability indicators,equipment available, etc.) and can produce results that implantconfigurations. For example, for a spinal surgery, the machine learningmodel can suggest pedicle screw configurations, e.g. characteristicssuch as screw diameter, length, threading and application parameterssuch as screw insertion point, angle, rotation speed, etc. As anotherexample, for an ACL replacement surgery, the machine learning model cansuggest graft type, attachment type (e.g. screw material, length, orconfiguration features), graft attachment locations, drill depths, etc.

In some implementations, the converted implant information can be usedin a multi-stage process for selecting aspects of an implantconfiguration. For example, for a spinal surgery, the multi-stageprocess can include method 800 or method 850, discussed below. Invarious steps of this these processes, either an algorithm can be usedto generate results for that step or a machine learning model, trainedfor that step, can be applied.

In some implementations, the procedure for identifying implantconfigurations for a spinal surgery can include processing implantsurgery information to locate targeted vertebrae and their pedicles inimages, on available axes; identifying and tagging vertebraecharacteristics; determining a preferred screw insertion point based ona mapping between tags and insertion point criteria (e.g. where themapping can be a representation of a medical definition of a pediclescrew insertion point—described below); performing measurements, on theimages, of the pedicle isthmus width and height and length of thepedicle and vertebral body, starting at the preferred insertion point;measuring the angle between the line used to determine length and thesagittal plane, in the axial view; and measuring the angle between thatlength line and the axial plane.

In some implementations, machine learning models can be trained toperform some of these tasks for identifying implant configurations. Forexample, machine learning models can be trained to identify particularvertebral pedicles in various images, which can then be atomicallymeasured and aggregated across images, e.g. storing minimal, maximal,median, or average values, as appropriate given the target beingmeasured. As another example, a machine learning model can receive theset of images and determine an order or can select a subset of theimages, for automatic or manual processing. In some implementations, amachine learning model can be used to localize and classify the targetwithin an image, such as by identifying a target vertebra or localizingthe end of the femur or meniscus edges. In some implementations, amachine learning model can be used to segment target vertebrae, femur,tibia, or other anatomical features in the target area, to determinetheir boundaries. In some implementations, a machine learning model canbe used to localize insertion points. In some implementations, a machinelearning model can be used to segment images to determine boundaries ofanatomical structures (e.g., boundaries of bones, organs, vessels,etc.), density of tissue, characteristics of tissue, or the like.

In various implementations, the results from the above stages can beused in inference formulae to compute the implant configurations. Forexample, a maximal screw diameter can be determined using the smallestpedicle isthmus dimension found across all the images of the targetvertebrae (which can be adjusted to include a safety buffer). As anotherexample, a maximal screw length can be determined using the smallestmeasurement of pedicle and vertebral body length, across all the targetvertebra in question (which can be adjusted to include a safety buffer).

Machine learning models, as used herein, can be of various types, suchas Convolutional Neural Networks (CNNs), other types of neural networks(e.g. fully connected), decision trees, forests of classification trees,Support Vector Machines, etc. Machine learning models can be trained toproduce particular types of results, as discussed below in relation toFIG. 9. For example, a training procedure can include obtaining suitabletraining items with input associated with a result, applying eachtraining item to the model, and updating model parameters based oncomparison of model result to training item result.

In some implementations, automated selection of implant configurationscan be limited to only cases likely to produce good results, e.g. onlyfor certain pathologies, types of patients, surgery targets (e.g. thepart of the spine that needs to be fused), or where confidence scoresassociated with machine learning model outputs are above a threshold.For example, in the spinal surgery example, automation can be limited tocommon types of spinal fusions, such as L3/L4, L4/L5, or L5/S1, certainpathologies such as spondylolisthesis or trauma, or patients withcertain characteristics, such as being in a certain age group. Asanother example, for an ACL replacement, automation can be limited tocases without other ligament tears.

At block 708, method 700 can provide results specifying one or morefeatures of an implant configuration. For example, the results for aspinal surgery can include selection of pedicle screw type anddimensions for each vertebra and guidance on an insertion point andangle for each screw. As another example, results for an ACL replacementsurgery can include selection of implant graft type, connection type,joint dimensions, and guidance on connection points such as drilllocations and depths. In some implementations, the results can bespecified in a natural language, e.g. using templates that can be filledin with recommendations. In some cases, the results from the analysis ofblock 706 can be mapped to particular reasons for the implantconfiguration recommendations, and these reasons can be supplied alongwith the recommendations.

In some implementations, the results can be based on medical definitionsof preferred implant configurations, where the preferred implantconfigurations can be mapped to a particular surgical target area basedon the results from block 706. For example, results for spinal surgerypedicle screws can include a preferred insertion point, e.g. defined,for lumbar vertebrae, at the intersection of the superior articularfacet, transverse process, and pars interarticularis; and for thoracicspine or cervical spine, at the intersection of the superior articularfacet plane and transverse process. As another example, a preferredscrew angle can be, in axial view, the angle between the sagittal planeand the line defined by the insertion point and midpoint of the pedicleisthmus. In sagittal view the preferred screw angle can be parallel tothe superior vertebral endplate. In addition, a maximal screw length canbe defined as the distance between the insertion point and the farcortical boundary of the vertebra, at a particular screw angle. Amaximal screw diameter can be the minimal width of the pedicle isthmus,on any axis. Each of these can be modified to include a certain safetybuffer, which can vary depending on the size of the vertebra. Theresults from block 706 can identify features of an individual patient,which can be used in conjunction with the foregoing implantconfiguration definitions to specify patient specific implantconfigurations, e.g. in natural language, as image annotations, in a 3Dmodel, as discussed below.

The implant configuration results can be specified in various formats.For example, results can be in a natural language, coordinates one ofvarious coordinate systems, as instructions for a robotic system, asannotations to one or more images, or as a virtual model of the surgerytarget area. The results can be used to augment the implant surgery inmultiple ways. For example, results can be added to a preoperative plan.Results can trigger acquisition of implant materials, such as by havingselected implants ordered automatically or having designs forpatient-specific screws provided for 3D-printing. As another example,results can be used to provide recommendations during a surgicalprocedure, e.g. with text or visual annotations provided as overlies ona flat panel display, through auditory or haptic feedback alerts, orusing an AR or VR system, e.g. to display an overlay of the implant onthe patient anatomy or to display guidance on the suggested insertionpoint and angle. In some implementations, the results can be used tocontrol robotic systems, e.g. causing a robotic arm to align itselfaccording to the recommended insertion point and angle, which may befirst confirmed by a surgeon.

The method 700 can be used in a wide range of procedures, e.g. openprocedures, minimally invasive procedures, orthopedic procedures,neurological procedures, reconstructive implants, maxillofacialprocedures (e.g., maxillary implants), or other procedure. In somesurgical procedures, the implant information at block 702 can includeimplant dimensions, material information (e.g., composition of implant),and images of the patient. At block 706, the implant configuration canbe implant dimensions (e.g., when in a delivery state or implantedstate), implant functionality, or the like.

FIG. 8A illustrates a flowchart showing a method 800 for applyinganalysis procedures that can utilize machine learning models, accordingto an embodiment. In some implementations, method 800 is performed as asub-process of block 706. At block 802, method 800 can receive implantsurgery information. This can be some of the converted implant surgeryinformation from block 704. In some implementations, the implant surgeryinformation can include one or more images of the surgery target area,e.g. MRI scans of a spine, X-rays of a wrist, ultrasound images of anabdomen, etc.

At block 804, method 800 can localize and classify a target in one ormore of the images of the surgery target area. In variousimplementations, this can be accomplished by applying a machine learningmodel trained for the particular target area to identify surgicaltargets or by finding a centroid point of each displayed vertebra,performing vertebral classification using image recognition algorithms,and determining whether the classified vertebrae match a list ofvertebrae identified for the surgery. In some implementations, if theimage does not contain at least one target of interest, the image can bedisregarded from further processing.

At block 806, method 800 can segment the identified target(s) from block804 to determine their boundaries. At block 808, method 800 can localizeimplant insertion points. In some implementations, blocks 806 and 808can be performed using machine learning models or algorithms, e.g. thatidentify particular patterns, changes in color, shapes, etc.

At block 810, method 800 can localize and segment individual targetfeatures. For example, in a spinal surgery where targets are vertebrae,at block 810 method 800 can identify vertebrae pedicles and theiristhmus, and measure these features. In some implementations, this canbe accomplished using a machine learning model trained to detect eachtype of feature. In some implementations, detecting the pedicle and theisthmus of vertebra from annotated images can include measuring theisthmus width and tracking the minimal value across images and planes,defining the angle between the line that passes through at least twomidpoints in the pedicle, and the reference plane, measuring the maximallength through that line, and tracking the minimal value acrossmeasurements. In some implementations, isthmus determination andmeasurement can be accomplished by starting at a point inside a pedicle,computing the distance to pedicle borders in multiple directions, takingthe minimum length. In other implementations, the isthmus determinationand measurement can be accomplished by scanning, for example usinghorizontal lines that intersect with pedicle borders in an axial view,and finding the minimum-length line.

In some implementations, the steps performed at any of blocks 804-810can be repeated for each of multiple target area images, aggregatingresults from the multiple images. For example, in a step for identifyingand measuring vertebrae pedicles, an aggregated measurement for aparticular pedicle can be the minimum measured width of the pedicle fromall of the images showing that particular pedicle.

At block 812, method 800 can use results from any of blocks 804-810 tocompute an implant configuration (e.g. characteristics and applicationparameters). For example, the minimum width of a pedicle found acrossthe images showing that pedicle, with some buffer added, can be theselected width characteristic of a pedicle screw implant. As anotherexample, a screw angle could be determined using an identified insertionpoint and a center of the pedicle isthmus, with respect to center axis,depending on the image plane. The angles in axial and sagittal planescan be either the median or average angles across the multiple images.As a further example, a maximal screw length can be determined as thelength of the line defined by the insertion point, the insertion angle,and the point where the line hits the cortical wall of the vertebra,minus some safety buffer. This length can be computed from multipleimages and the minimum across all the images can be used for of thisscrew.

FIG. 8B illustrates a flowchart showing a method 850 for applyinganalysis procedures that can utilize virtual models, according to anembodiment. In some implementations, method 850 is performed as asub-process of block 706. At block 852, method 850 can receive implantsurgery information. This can be some of the converted implant surgeryinformation from block 704. In some implementations, the implant surgeryinformation can include one or more images of the surgery target area.

At block 854, method 850 can build one or more virtual models of thetarget surgery area based on the images and/or other measurement data inthe implant surgery information. A virtual model, as used herein, is acomputerized representation of physical objects, such as the target areaof a surgery (e.g. portions of a patient's spine) and/or implants (e.g.screws, rods, etc.). In some implementations, virtual models can beoperated according to known physical properties, e.g. reactions toforces can be predicted according to known causal relationships. Invarious implementations, the virtual models generated by method 850 canbe two-dimensional models or three-dimensional models. For example, atwo-dimensional model can be generated by identifying portions of animage as corresponding to parts of a patient's anatomy, such that acomputing system can determine how implant characteristics would fit inrelation to the determined anatomy parts. As another example, athree-dimensional model can be generated by identifying shapes andfeatures in individual images, from a set of successive images, andmapping the identified shapes and features into a virtualthree-dimensional space, using relationships between images. Finiteelement analysis techniques can be used to predict stresses, strains,pressures, facture, and other information and be used to designimplants, surgical tools, surgical techniques, etc. For example, theimplant configuration can be determined based on predetermined stresses(e.g., maximum allowable stresses in the tissue and/or implant, yieldstrength of anatomical structures and/or implant components, etc.),fracture mechanics, or other criteria defined by the physician orautomatically determined based on, for example, tissue characteristics,implant design, or the like. In some embodiments, fatigue life can bepredicted using stress or strain based techniques.

A virtual model can also analyze mechanical interaction between apatient's vertebrae, loading of implants, and other devices (e.g., rods,ties, brackets, plates, etc.) coupled to those implants. The output ofthese analyses can be used to select pedicle screw configurations,insertion trajectories, and placement location to optimize screwpull-out strength, maximum allowable loading (e.g., axial loads, shearloads, moments, etc.) to manage stresses between adjacent vertebrae, ormaximum allowable stress in regions of the bone at risk for fracture.

In some embodiments, a user could identify areas of weakened bone orareas on images of the patient where there is risk of a fracture due tothe presence of a screw or other implant. This information can beprovided to the virtual model. The virtual model can be used to evaluatewhether the configuration or location of the implant would create anunacceptable risk of fracture in the identified region. If so, thesystem could alert the user to that risk or modify the implantconfiguration or the procedure to reduce the risk to an acceptablelevel. In other embodiments, the system could identify these areas ofhigh fracture risk automatically. In yet another embodiment, the systemcould provide data to the user such as the maximum torque to apply to agiven pedicle screw during the surgical procedure such that tissuetrauma, risk of fracture, or adverse advents is minimized.

At block 856, method 850 can localize and classify areas of interestwithin the virtual model(s) from block 854. This can be accomplishedusing object recognition that matches shapes of known objects to shapeswithin the virtual models. For example, in a virtual model for a spinalsurgery, the images can be MRI images of vertebrae. The virtualvertebrae can be labeled (e.g. c1-s5) and virtual model vertebraecorresponding to the vertebrae for which the spinal procedure is plannedcan be selected as the areas of interest. In some implementations,additional areas around the selected areas can be added to the areas ofinterest, allowing the surgeon to select alternative options before orduring the procedure. For example, the one or two vertebrae adjacent, onone or both sides, to the planned vertebrae can be additionallyselected.

At block 858, method 850 can segment the areas of interest, identifiedat block 856, to determine various boundaries and other features, suchas the pedicle boundaries and the pedicle isthmus. In someimplementations, the segmentation or boundary determinations can beperformed using a machine learning model. The machine learning model canbe trained, for the type of implant surgery to be performed, to receivea portion of a virtual model and identify target portion segmentationsor boundaries.

At block 860, method 850 can localize an insertion point for the implantin the target area. In some implementations, this can be accomplished byapplying a machine learning model trained to identify insertion points.In some implementations, localizing insertion points can be accomplishusing an algorithm, e.g. that identify particular patterns, changes incolor, shapes, etc. identified as corresponding to preferred implantinsertion points.

At block 862, method 850 can compute an implant configuration based onthe virtual model(s) and/or determinations made in blocks 856-860. Insome implementations, the implant can be associated with requirementsfor their application and properties to maximize or minimize. In thesecases, the implant configuration can be specified as the configurationthat fits with the virtual model, achieving all the requirements, andoptimizing the maximizable or minimizable properties. For example, whenmethod 850 is performed to determine pedicle screw configurations for aspinal surgery, virtual pedicle screws can be placed in a virtual modelgenerated at block 854, according to the insertion points determined atblock 860. The virtual pedicle screws can further be placed to: notbreach cortical vertebral boundaries (e.g. determined at block 858),with a specified amount of buffer, while maximizing the screw diameterand length, taking into consideration required buffers and close tooptimal insertion angle, defined by the pedicle isthmus center andinsertion point, for each vertebra (e.g. determined at block 858). Insome implementations, this placement of the implant can be performed asa constraint optimization problem. For example, a virtual screw can beplaced inside the segmented vertebral body in the virtual model. Theplacement can then be adjusted until an equilibrium is reached thatoptimizes the parameters while conforming to the implant constraints.For example, method 850 can maximizing screw diameter and length whilealigning with an optimal angle and avoiding cortical breaches.

FIG. 9 illustrates a flowchart showing a method for training a machinelearning model, according to an embodiment. Machine learning models,such as neural networks, can be trained to produce types of results. Aneural network can be trained by obtaining, at block 902, a quantity of“training items,” where each training item includes input similar toinput the model will receive when in use and a corresponding scoredresult. At block 904, the input from each training item can be suppliedto the model to produce a result. At block 906, the result can becompared to the scored result. At block 908, model parameters can thenbe updated, based on how similar the model result is to the scoredresult and/or whether the score is positive or negative.

For example, a model can be trained using sets of pre-operative MRIscans of vertebrae paired with pedicle screw placements used in thesurgery and corresponding scores for the result of that surgery. Theimages can be converted to arrays of integers that, when provided to themachine learning model, produce values that specify screw placements.The screw placements can be compared to the actual screw placement usedin the surgery that produced the training item. The model parameters canthen be adjusted so the model output is more like the screw placementused in the surgery if the surgery was a success or less like the screwplacement used in the surgery if the surgery was a failure. The amountof adjustment to the model parameters can be a function of how differentthe model prediction was from the actual screw configuration used and/orthe level of success or failure of the surgery.

As discussed above, machine learning models for the surgical assistancesystem can be trained to produce various results such as: to directlyproduce implant configurations upon receiving implant surgeryinformation, to identify particular vertebral pedicles in variousimages, to determine an order or subset of images for processing, tolocalize and classify the target within an image, to segment targetvertebrae, to determine boundaries or other features, to localizeinsertion points, etc.

In various implementations, the training data for a machine learningmodel can include input data such as medical imaging data, other patientdata, or surgeon data. For example, model input can include images ofthe patient, patient sex, age, height, weight, type of pathology,occupation, activity level, etc., specifics of implant systems (e.g.types and dimensions), availability of available implants, or aspects ofa surgeon's preoperative plan (e.g. surgeon's initial implantconfiguration, detection and measurement of the patient's anatomy onimages, etc.) In some implementations, model training data input caninclude surgeon specifics, such as statistics or preferences for implantconfigurations used by the surgeon performing the implant surgery oroutcomes for implant usages. For example, surgeons may have better skillor experience with particular implant configurations, and the system canbe trained to select implant configurations the particular surgeon ismore likely to use successfully. The training data input can be pairedwith results to create training items. The results can be, for example,human annotated medical imaging data (as a comparison foridentifications such as boundaries and insertion points identified by amodel), human feedback to model outputs, surgeons' post-operativesuggestion feedback (e.g. whether the surgeon accepted model providedrecommendations completely, or made certain changes, or disregarded),surgeons post-operative operation outcome success score, post-operativeimages that can be analyzed to determine results, the existence ofcertain positive or negative patient results, such as cortical breachesor other complications that might have occurred in the procedure,overall level of recovery, or recovery time.

In an illustrative embodiment, any of the operations, processes, etc.described herein can be implemented as computer-readable instructionsstored on a computer-readable medium. The computer-readable instructionscan be executed by a processor of a mobile unit, a network element,and/or any other computing device.

There is little distinction left between hardware and softwareimplementations of aspects of systems; the use of hardware or softwareis generally (but not always, in that in certain contexts the choicebetween hardware and software can become significant) a design choicerepresenting cost vs. efficiency tradeoffs. There are various vehiclesby which processes and/or systems and/or other technologies describedherein can be effected (e.g., hardware, software, and/or firmware), andthat the preferred vehicle will vary with the context in which theprocesses and/or systems and/or other technologies are deployed. Forexample, if an implementer determines that speed and accuracy areparamount, the implementer may opt for a mainly hardware and/or firmwarevehicle; if flexibility is paramount, the implementer may opt for amainly software implementation; or, yet again alternatively, theimplementer may opt for some combination of hardware, software, and/orfirmware.

The foregoing detailed description has set forth various embodiments ofthe devices and/or processes via the use of block diagrams, flowcharts,and/or examples. Insofar as such block diagrams, flowcharts, and/orexamples contain one or more functions and/or operations, it will beunderstood by those within the art that each function and/or operationwithin such block diagrams, flowcharts, or examples can be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment,several portions of the subject matter described herein may beimplemented via Application Specific Integrated Circuits (ASICs), FieldProgrammable Gate Arrays (FPGAs), digital signal processors (DSPs), orother integrated formats. However, those skilled in the art willrecognize that some aspects of the embodiments disclosed herein, inwhole or in part, can be equivalently implemented in integratedcircuits, as one or more computer programs running on one or morecomputers (e.g., as one or more programs running on one or more computersystems), as one or more programs running on one or more processors(e.g., as one or more programs running on one or more microprocessors),as firmware, or as virtually any combination thereof, and that designingthe circuitry and/or writing the code for the software and or firmwarewould be well within the skill of one of skill in the art in light ofthis disclosure. In addition, those skilled in the art will appreciatethat the mechanisms of the subject matter described herein are capableof being distributed as a program product in a variety of forms, andthat an illustrative embodiment of the subject matter described hereinapplies regardless of the particular type of signal bearing medium usedto actually carry out the distribution. Examples of a signal bearingmedium include, but are not limited to, the following: a recordable typemedium such as a floppy disk, a hard disk drive, a CD, a DVD, a digitaltape, a computer memory, etc.; and a transmission type medium such as adigital and/or an analog communication medium (e.g., a fiber opticcable, a waveguide, a wired communications link, a wirelesscommunication link, etc.).

Those skilled in the art will recognize that it is common within the artto describe devices and/or processes in the fashion set forth herein,and thereafter use engineering practices to integrate such describeddevices and/or processes into data processing systems. That is, at leasta portion of the devices and/or processes described herein can beintegrated into a data processing system via a reasonable amount ofexperimentation. Those having skill in the art will recognize that atypical data processing system generally includes one or more of asystem unit housing, a video display device, a memory such as volatileand non-volatile memory, processors such as microprocessors and digitalsignal processors, computational entities such as operating systems,drivers, graphical user interfaces, and applications programs, one ormore interaction devices, such as a touch pad or screen, and/or controlsystems including feedback loops and control motors (e.g., feedback forsensing position and/or velocity; control motors for moving and/oradjusting components and/or quantities). A typical data processingsystem may be implemented utilizing any suitable commercially availablecomponents, such as those typically found in datacomputing/communication and/or network computing/communication systems.

The herein described subject matter sometimes illustrates differentcomponents contained within, or connected with, different othercomponents. It is to be understood that such depicted architectures aremerely examples, and that in fact many other architectures can beimplemented which achieve the same functionality. In a conceptual sense,any arrangement of components to achieve the same functionality iseffectively “associated” such that the desired functionality isachieved. Hence, any two components herein combined to achieve aparticular functionality can be seen as “associated with” each othersuch that the desired functionality is achieved, irrespective ofarchitectures or intermedial components. Likewise, any two components soassociated can also be viewed as being “operably connected”, or“operably coupled”, to each other to achieve the desired functionality,and any two components capable of being so associated can also be viewedas being “operably couplable”, to each other to achieve the desiredfunctionality. Specific examples of operably couplable include but arenot limited to physically mateable and/or physically interactingcomponents and/or wirelessly interactable and/or wirelessly interactingcomponents and/or logically interacting and/or logically interactablecomponents.

From the foregoing, it will be appreciated that various embodiments ofthe present disclosure have been described herein for purposes ofillustration, and that various modifications may be made withoutdeparting from the scope and spirit of the present disclosure.Accordingly, the various embodiments disclosed herein are not intendedto be limiting.

What is claimed is:
 1. A method for providing assistance for a spinal surgery, the method comprising: providing information for types of spinal screws and sizes of the spinal screws, wherein at least one spinal screw of the spinal screws is used as an implant during the spinal surgery; defining areas of interest in diagnostic data of a patient and a screw bone type; identifying salient features of the areas of interest in the diagnostic data; determining, based on the salient features, a first angle, a second angle, and a position entry point for the at least one spinal screw to be used as the implant; determining, based on the salient features, a maximum screw diameter and a length of the at least one spinal screw; and identifying and suggesting, based on the maximum screw diameter, the length of the at least one spinal screw, and the screw bone type, the at least one spinal screw to be implanted during the spinal surgery.
 2. The method of claim 1, wherein the diagnostic data comprises Magnetic Resonance Imaging (MRI), Computed Tomography (CT), or x-ray images of an XY plane and an XZ plane of the patient's spine.
 3. The method of claim 1, wherein determining the maximum screw diameter and the length are further based on the first angle, the second angle, and position entry point.
 4. The method of claim 1, further comprising: identifying at least one spinal abnormality by: automatically measuring a distance between a selected first salient feature of one vertebra and a selected second salient feature of another vertebra; based on the measuring, identifying one or more disk pinches or bulges; and based on the identified disk pinches or bulges, identifying one or more herniated disks; and for each particular identified at least one spinal abnormality, causing a display to graphically identify the particular spinal abnormality.
 5. The method of claim 1, further comprising: providing a design interface for altering the first angle, the second angle, the position entry point, or the suggested at least one spinal screw; and receiving, via the design interface, modifications from a surgeon for the first angle, the second angle, the position entry point, or the suggested at least one spinal screw.
 6. The method of claim 5, wherein the modifications from the surgeon comprise a modification to the first angle, the second angle, or the position entry point; and wherein the method further comprises automatically changing the suggested at least one spinal screw to account for the modifications from the surgeon.
 7. The method of claim 5, further comprising determining a spinal surgical plan based on at least one of the identified salient features, the length of the at least one spinal screw, and screw bone type.
 8. A computer-implemented method for providing surgical assistance, the method comprising: receiving diagnostic data of a patient; analyzing the diagnostic data to identify one or more areas of interests for a surgery to be performed; using at least one trained machine learning model to determine, based on the analysis of the diagnostic data and implant information, a patient-specific implant configuration for an implant; and determining patient-specific surgical information for implanting the implant in the patient.
 9. The method of claim 8, further comprising using the at least one trained machine learning model to determine the patient-specific surgical information.
 10. The method of claim 8, further comprising causing the patient-specific surgical information to be sent for a physician, wherein the patient-specific information includes a surgical plan for delivering the implant to a site within the patient.
 11. The method of claim 8, further comprising causing the patient-specific surgical information to be sent to at least one of a navigation system or a robotic surgery system.
 12. The method of claim 8, wherein the trained machine learning model was trained based on a set of patient data.
 13. The method of claim 8, further comprising: analyzing one or more images while the implant is within the patient's body; and determining additional patient-specific surgical information based on the analysis of the one or more images.
 14. The method of claim 8, further comprising: training the at least one machine learning model based on a set of patient data; and identifying at least one anatomical feature in the one or more areas of interest using at least one trained machine learning model.
 15. The method of claim 8, further comprising: generating, based on the diagnostic data, a three-dimensional virtual model; and performing one or more structural analyses based on the three-dimensional virtual model to determine the one or more patient-specific implant configuration and/or patient-specific surgical information.
 16. The method of claim 8, further comprising training the machine learning model based on one or more analyses of a two-dimensional or three-dimensional virtual model.
 17. The method of claim 8, wherein analyzing the diagnostic data comprises boundary detection, edge detection, tissue identification, structural analysis, tissue density, and/or feature matching.
 18. The method of claim 8, further comprising identifying, based on the patient data, at least one of an implantation site, anatomical landmark, or anatomical abnormality.
 19. The method of claim 8, further comprising automatically analyzing the diagnostic data, determining the patient-specific implant configuration, and determining the patient-specific surgical information.
 20. The method of claim 8, further comprising: receiving physician input; and automatically, based on the physician input, determining the patient-specific implant configuration and determining patient-specific surgical information.
 21. A method, comprising: obtaining implant surgery information, for an implant surgery, the implant surgery information comprising at least a set of images; converting the implant surgery information for analysis, wherein the conversion includes transforming at least some of the set of images into data structures for one or more machine learning models; applying analysis procedures using the converted implant surgery information, wherein the analysis procedures apply at least one of the one or more machine learning models to: automatically identify specified physical features of a target of the implant surgery; automatically segment the target of the implant surgery to determine its boundaries; automatically identify an implant insertion point on the target of the implant surgery; or any combination thereof; and providing results indicating a suggested implant configuration based on outcomes of the applied analysis procedures.
 22. The method of claim 21, wherein the implant insertion point is located along a vertebra.
 23. The method of claim 21, wherein the implant insertion point is between adjacent anatomical features.
 24. The method of claim 21, further comprising periodically training the one or more machine learning models.
 25. A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations for generating an implant configuration, the operations comprising: obtaining implant surgery information, for an implant surgery, the implant surgery information comprising at least a set of images; converting the implant surgery information for analysis, wherein the conversion includes transforming at least some of the set of images into data structures for one or more machine learning models; applying analysis procedures using the converted implant surgery information, wherein the analysis procedures apply at least one of the one or more machine learning models to: automatically identify specified physical features of a target of the implant surgery; automatically segment the target of the implant surgery to determine its boundaries; automatically identify an implant insertion point on the target of the implant surgery; or any combination thereof; and providing results indicating a suggested implant configuration based on outcomes of the applied analysis procedures.
 26. The computer-readable storage medium of claim 25, wherein the set of images comprise one or more of: Magnetic Resonance Imaging (MRI), Computed Tomography (CT), x-ray images, or any combination thereof.
 27. The computer-readable storage medium of claim 25, wherein the implant surgery is a spinal surgery; and wherein the applied at least one of the one or more machine learning models is a machine learning model trained to automatically identify the specified physical features by localizing pedicles of target vertebrae.
 28. The computer-readable storage medium of claim 27, wherein the applied at least one of the one or more machine learning models further includes a machine learning model trained to automatically identify the isthmus of the target vertebrae.
 29. The computer-readable storage medium of claim 27, wherein the operations further comprise automatically identifying the isthmus of the target vertebrae by: selecting a point inside a localized pedicle; taking multiple measurements from the point to an identified border of the localized pedicle; and selecting the measurement, of the multiple measurements, that is the smallest.
 30. The computer-readable storage medium of claim 25, wherein the implant surgery is a spinal surgery and the target of the implant is a set of target vertebrae; wherein the application of the analysis procedures identifies multiple pedicle measurements for the target vertebrae; and wherein the suggested implant configuration includes an indication of a pedicle screw diameter that is based on the minimum of the multiple pedicle measurements.
 31. The computer-readable storage medium of claim 25, wherein the implant surgery is a spinal surgery and the target of the implant is a set of target vertebrae; wherein the application of the analysis procedures identifies a pedicle isthmus for at least a particular vertebra of the target vertebrae; and wherein the suggested implant configuration includes an indication of a pedicle screw angle that is based on the pedicle isthmus.
 32. The computer-readable storage medium of claim 25, wherein the implant surgery is a spinal surgery and the target of the implant is a set of target vertebrae; wherein the application of the analysis procedures identifies an insertion point and an insertion angle; and wherein the suggested implant configuration includes an indication of a pedicle screw length that is based on the insertion point, the insertion angle, and a point at which a line defined by the insertion point and insertion angle intersects with a cortical wall.
 33. The computer-readable storage medium of claim 25, wherein the implant surgery is a spinal surgery, the target is a set of target vertebrae, and the application of the analysis procedures comprises: generating a virtual model, based on at least the set of images, wherein the virtual model includes at least a representation of the target vertebrae; placing, in the virtual model, a virtual pedicle screw in relation to at least one of the target vertebrae; and adjusting a configuration of the virtual pedicle screw including at least a size of the virtual pedicle screw and a placement of the virtual pedicle screw to optimize a set of implant parameters while maintaining each of a set of implant requirements.
 34. The computer-readable storage medium of claim 33, wherein the implant requirements comprise not breaching any of a set of boundaries that were established by applying a machine learning model.
 35. The computer-readable storage medium of claim 33, wherein the implant parameters comprise maximizing virtual pedicle screw diameter and length.
 36. A system for generating an implant configuration, the system comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising: obtaining implant surgery information, for an implant surgery, the implant surgery information comprising at least a set of images; converting the implant surgery information for analysis; applying analysis procedures using the converted implant surgery information, wherein the analysis procedures apply at least one of the one or more machine learning models to: automatically identify specified physical features of a target of the implant surgery; automatically segment the target of the implant surgery to determine its boundaries; automatically identify an implant insertion point on the target of the implant surgery; or any combination thereof; and providing results indicating a suggested implant configuration based on the applied analysis procedures.
 37. The system of claim 36, wherein the implant surgery is a spinal surgery; and wherein the applied at least one of the one or more machine learning models is a machine learning model trained to localize pedicles of target vertebrae.
 38. The system of claim 36, wherein the implant surgery is a spinal surgery, the target is a set of target vertebrae, and the application of the analysis procedures comprises: generating a virtual model, based on at least the set of images, wherein the virtual model includes at least a representation of the target vertebrae; placing, in the virtual model, a virtual pedicle screw in relation to at least one of the target vertebrae; and adjusting a configuration of the virtual pedicle screw including at least a size of the virtual pedicle screw and a placement of the virtual pedicle screw to optimize a set of implant parameters while maintaining each of a set of implant requirements; wherein the implant requirements comprise not breaching any of a set of boundaries that were established by applying a machine learning mode; and wherein the implant parameters comprise maximizing virtual pedicle screw diameter and length. 